Implementation of Efficient Multilayer Perceptron ANN Neurons on Field Programmable Gate Array Chip

Artificial Neural Network is widely used to learn data from systems for different types of applications. The capability of different types of Integrated Circuit (IC) based ANN structures also depends on the hardware backbone used for their implementation. In this work, Field Programmable Gate Array (FPGA) based Multilayer Perceptron Artificial Neural Network (MLP-ANN) neuron is developed. Experiments were carried out to demonstrate the hardware realization of the artificial neuron using FPGA. Two different activation functions (i.e. tan-sigmoid and log-sigmoid) were tested for the implementation of the proposed neuron. Simulation result shows that tan-sigmoid with a high index (i.e. k >= 40) is a better choice of sigmoid activation function for the harware implemetation of a MLP-ANN neuron. Index Term-ANN, ASIC, DSP, FPGA, MLP 1.0 INTRODUCTION An artificial neuron was inspired principally from the structure and functions of the biological neuron. It learns through an iterative process of adjustment of its synaptic weights and a neuron becomes more knowledgeable after each iteration of the learning process. The ultimate aim of learning by the neuron is to adjust the weights and update the output for a new actual output which coincides with the desired output. However, the capability of a single artificial neuron is very limited. For instance, the Perceptron (a threshold neuron) cannot learn non-linearly separable function [1]. To learn functions that cannot be learned by a single neuron, an interconnection of multiple neurons called Neural Network (NN) or Artificial Neural Network (ANN) must be employed. Apart from the artificial neuron which is the basic processing units in ANN, there are patterns of connections between the neurons and the propagation of data called network topology. There are two main types of ANN topology which are; feedforward and recurrent network topologies. In feed-forward networks, the data flow from input to output strictly in a forward direction and there is no feedback of connections while in recurrent networks, there are feedback connections. A commonly used feed-forward network topology is MultiLayer Perceptron (MLP). MLP caters for learning of nonlinear functions and Figure 1.0 shows its architectural representation. Fig. 1.0. Multi-Layer Perceptron (MLP) topology [2]. The MLP networks are typically trained with the training algorithm called the Backpropagation (BP) algorithm which is a supervised learning method that maps the process inputs to the desired outputs by minimizing the errors between the desired outputs and the calculated outputs [2]. BP is an application of the gradient method or other numerical Output layer